Image generation apparatus and method for generating plurality of images with different resolution and/or brightness from single image

ABSTRACT

Provided is an image generation system and method, more particularly, an image generation system and method which can generate a high dynamic range image from a plurality of images acquired in a single short exposure. The image generation system To includes an image generation system having an image acquisition unit to acquire an image, an image generation unit to generate, from the acquired image, a plurality of images with different resolution and brightness, and an image synthesis unit to synthesize the generated images.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of U.S. application Ser.No. 11/826,165, filed on Jul. 12, 2007, in the U.S. Patent and TrademarkOffice, which claims priority of Korean Patent Application No.10-2006-0085772, filed on Sep. 6, 2006, in the Korean IntellectualProperty Office, the entire disclosures of which are incorporated hereinby reference.

BACKGROUND

1. Field

One or more embodiments of the present invention relate to an imagegeneration system, method and medium, and more particularly, to an imagegeneration system, method and medium which can acquire a plurality ofimages with different luminance in a single photographing operation andgenerate a high dynamic range image from the acquired images.

2. Description of the Related Art

When image photographing apparatuses, such as digital cameras andcamcorders, photograph a subject against background light, they generateimages having different exposures in order to enhance image sensitivity.Then, the image photographing apparatuses synthesize the generatedimages and provide the synthesized image to a user.

Specifically, when an identical subject is photographed using differentexposure times, the shorter the exposure time, the higher the contrastof the image and thus the clearer the edges of the image. In addition,the longer the exposure time, the clearer the color of the image.Therefore, if an image acquired using a long exposure time and an imageacquired using a short exposure time are synthesized, a high dynamicrange image can be obtained.

Referring to FIG. 1, a conventional image photographing apparatusgenerates a first image 11 acquired using a short exposure time, and asecond image 12 acquired using a longer exposure time relative to thefirst image 11, synthesizes the first and second images 11 and 12, andgenerates a high dynamic range image 13.

In order to acquire images with different luminance through multipleexposures, a user has to maintain the same posture for a predeterminedperiod of time, which is usually inconvenient to the user. Furthermore,if the user's hands tremble or shake while maintaining the same posture,image blurring occurs. Alternatively, even if the user can maintain thesame posture for the predetermined period of time, if the position of asubject is changed while the user is taking photographs of the subject,the images acquired through the photographing operation are notidentical. Consequently, it is difficult to generate a normal image.

A method of changing structures, such as modifying an array of colorfilters or varying sensitivity of the color filters, has been used so asto eliminate the inconvenience of users having to the maintain the sameposture for a predetermined period of time. However, the method requiresadditional processing, such as demosaicing of the color filters andwhite balancing.

U.S. Patent Publication No. 2006-0066750 discloses an image sensor whichcan generate a wide dynamic range image by exposing odd-numbered columnsof pixels in a pixel array for a short period of time, exposingeven-numbered columns of pixels in the pixel array for a longer periodof time than the odd-numbered columns of pixels, combining theodd-numbered columns of pixels and the even-numbered columns of pixelsinto respective pairs, and outputting the respective pairs of theodd-numbered columns of pixels and the even-numbered columns of pixels.However, the conventional art fails to suggest a method of eliminatingthe inconvenience to users of having to maintain the same posture for along exposure time, or a technique for preventing image blurring due tothe trembling hands of the users.

In this regard, a method of generating a high dynamic range image usingimages acquired in a single short exposure is required.

SUMMARY

One or more embodiments of the present invention provide an imagegeneration system, method and medium, which can generate a high dynamicrange image from a plurality of image acquired in a single, shortexposure.

Additional aspects and/or advantages of the invention will be set forthin part in the description which follows and, in part, will be apparentfrom the description, or may be learned by practice of the invention.

To achieve at least the above and/or other aspects and advantages,embodiments of the present invention include an image generation systemincluding, an image acquisition unit to acquire an image, an imagegeneration unit to generate, from the acquired image, a plurality ofimages with different resolution and brightness, and an image synthesisunit to synthesize the generated images.

To achieve at least the above and/or other aspects and advantages,embodiments of the present invention include an image generation methodincluding, acquiring an image, generating, from the acquired image, aplurality of images with different resolution and brightness, andsynthesizing the generated images.

To achieve at least the above and/or other aspects and advantages,embodiments of the present invention include an image generation methodincluding, generating a plurality of images with different resolutionand brightness from an acquired image, and synthesizing the generatedimages.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects and advantages of the invention will becomeapparent and more readily appreciated from the following description ofthe embodiments, taken in conjunction with the accompanying drawings ofwhich:

FIG. 1 illustrates a conventional image generation process;

FIG. 2 illustrates an image generation system, according to a firstembodiment of the present invention;

FIG. 3 illustrates an example of a first image generated by a firstimage generation unit, such as illustrated in FIG. 2, according to anembodiment of the present invention;

FIG. 4 illustrates an example of a second image generated by a secondimage generation unit, such as illustrated in FIG. 2, according to anembodiment of the present invention;

FIG. 5 illustrates an exposure time-brightness function, according to anembodiment of the present invention;

FIG. 6 is a flowchart illustrating an image generation method used bythe image generation system, such as of FIG. 2, according to anembodiment of the present invention;

FIG. 7 illustrates a first synthesis method, according to an embodimentof the present invention;

FIG. 8 illustrates a process of generating a final image using the firstsynthesis method of FIG. 7, according to an embodiment of the presentinvention;

FIG. 9 illustrates a second synthesis method, according to an embodimentof the present invention;

FIG. 10 illustrates a process of generating a final image using thesecond synthesis method of FIG. 9, according to an embodiment of thepresent invention;

FIG. 11 illustrates a third synthesis method, according to an embodimentof the present invention;

FIG. 12 illustrates a process of generating a final image using thethird synthesis method of FIG. 11, according to an embodiment of thepresent invention;

FIG. 13 illustrates an image generation system, according to a secondembodiment of the present invention; and

FIG. 14 illustrates an image generation method, such as used by theimage generation system of FIG. 13, according to an embodiment of thepresent invention.

DETAILED DESCRIPTION

Reference will now be made in detail to embodiments of the presentinvention, examples of which are illustrated in the accompanyingdrawings, wherein like reference numerals refer to the like elementsthroughout. Embodiments are described below to explain the presentinvention by referring to the figures.

FIG. 2 illustrates an image generation system 100, according to anembodiment of the present invention. The image generation system 100 maybe understood to include any type of apparatus, such as a digitalcamera, a camcorder, a cellular phone, or a personal digital assistant(PDA) which may include a module for photographing an image (a stillimage or a moving image). Hereinafter, embodiments may be described inwhich the image generation system 100 is a digital camera, which canphotograph an image of a subject, as an example.

Specifically, referring to FIG. 2, the image generation system 100 mayinclude, for example, a calculation unit 110, an exposure adjustmentunit 120, an image acquisition unit 130, an image generation unit 140,an image synthesis unit 150, and an image output unit 160.

The calculation unit 110 may calculate an exposure time required for theimage acquisition unit 130, which will be described in greater detaillater, to acquire an image.

The exposure adjustment unit 120 may adjust a shutter based on theexposure time calculated by the calculation unit 110. In the case of acamera sensor without a mechanical shutter, an optical sensor may adjustthe time spent on receiving light. Therefore, an image acquired by theimage acquisition unit 130, which will be described in greater detaillater, may have the appropriate brightness. The present embodiment isdescribed using an example in which the image generation unit 100 is adigital camera. However, this is merely an example used to help promotethe understanding of embodiments of the present invention, and thepresent invention is not limited thereto. In other words, the presentinvention can be applied, in a similar manner, to any apparatus that canphotograph a still image or a moving image.

The image acquisition unit 130 may acquire an image of a subject basedon an amount of light received during the exposure time calculated bythe calculation unit 110. The image acquired by the image acquisitionunit 130 may be provided to the image generation unit 140, which willhereinafter be described in greater detail.

The image generation unit 140 may generate a plurality of images havingdifferent resolution and brightness from the image acquired by the imageacquisition unit 130. To this end, the image generation unit 140 mayinclude a first image generation unit 141 generating a first image and asecond image generation unit 142 generating a second image with lowerresolution than the first image.

Specifically, the first image generation unit 141 may generate the firstimage from the image acquired by the image acquisition unit 130. In thepresent embodiment, the first image may be understood as the imageacquired by the image acquisition unit 130. The first image may haveresolution corresponding to the number of pixels included in a CMOS(complementary metal-oxide semiconductor) image sensor (CIS) of theimage generation system 100. For example, it may be assumed that colorfilters r1 through r4, g1 through g8 and b1 through b4 are formed ineach pixel included in the CIS of the image generation system 100, asillustrated in FIG. 3. In this case, the first image may have resolutioncorresponding to the color filters r1 through r4, g1 through g8 and b1through b4. Here, r, g and b may be understood as red, green and bluefilters, respectively.

The second image generation unit 142 may generate the second image withlower resolution than the first image, from the first image.Specifically, the second image generation unit 142 may generate thesecond image, having lower resolution than the first image, from thefirst image using pixels, which use an identical color filter, as asingle pixel. The resolution of the second image may be represented bythe resolution of the first image as in Equation (1) below.

$\begin{matrix}{{{{Resolution}\mspace{14mu}{of}\mspace{14mu}{second}\mspace{14mu}{image}} = {{resolution}\mspace{14mu}{of}\mspace{14mu}{first}\mspace{14mu}{image} \times \frac{1}{N}}},} & {{Equation}\mspace{14mu} 1}\end{matrix}$

where N may indicate the number of pixels of the first image included ineach pixel of the second image. For example, if N=4, the second imagegeneration unit 142 may generate the second image having a quarter ofthe pixels included in the first image. In this case, the second imagemay have resolution corresponding to R1, G1, G2 and B1. R1 may beunderstood as r1+r2+r3+r4, G1 may be understood as g1+g2+g5+g6, G2 maybe understood as g3+g4+g7+g8, and B1 may be understood as b1+b2+b3+b4.In addition, R, G and B may be understood as red, green and bluefilters, respectively.

As described above, since each pixel of the second image may consist ofa plurality of pixels using an identical color filter in the firstimage, the second image may be brighter than the first image.Specifically, the brightness of the second image may be increased by thenumber of pixels of the first image included in each pixel of the secondimage, as compared with the brightness of the first image. For example,as described above, if each pixel of the second image consists of fourpixels of the first image, the brightness of the second image may beincreased to four times that of the first image.

According to an embodiment of the present invention, informationregarding the resolution and appropriate brightness of the second imagemay be predetermined. In this case, the calculation unit 110 maycalculate an exposure time (hereinafter referred to as a “first exposuretime”) for acquiring the first image based on an exposure time(hereinafter referred to as a “second exposure time”) for acquiring thesecond image with predetermined brightness and resolution of the secondimage. The first exposure time is given by Equation (2).

$\begin{matrix}{{T_{1} = \frac{T_{2}}{N}},} & {{Equation}\mspace{14mu} 2}\end{matrix}$

where T1 may indicate the first exposure time and T2 may indicate thesecond exposure time. In addition, N may indicate the number of pixelsof the first image included in each pixel of the second image, as, forexample, in Equation (1).

In Equation (2), the second exposure time T₂ may be calculated using anexposure time-brightness function. FIG. 5 illustrates an exposuretime-brightness function according to an embodiment of the presentinvention. The exposure time-brightness function may define therelationship between exposure time and brightness of an image. In theexposure time-brightness function, a horizontal axis indicates anexposure time, and a vertical axis indicates brightness of the imagecorresponding to the exposure time. Therefore, if brightness Y₂ of thesecond image is predetermined, the second exposure time T₂ for acquiringthe second image with the predetermined brightness Y₂ may be calculatedusing the exposure time-brightness function.

The image synthesis unit 150 may synthesize the first image generated bythe first image generation unit 141 and the second image generated bythe second image generation unit 142 and may generate a final image.Here, the first and second images may be synthesized using a variety ofmethods.

In an embodiment of the present invention, the image synthesis unit 150may increase the resolution of the second image to that of the firstimage and synthesize the first image and the second image with increasedresolution (hereinafter referred to as “first synthesis”). Here, theimage synthesis unit 150 may increase the resolution of the second imageto that of the first image through, for example, linear interpolation. Afinal image generated through the first synthesis may be displayed onthe image output unit 160 without additional processing processes.

In another embodiment of the present invention, the image synthesis unit150 may reduce the resolution of the first image to that of the secondimage and synthesize the second image and the first image with reducedresolution (hereinafter referred to as “second synthesis”). Since afinal image generated through the second synthesis has resolutionreduced to that of the second image, the image synthesis unit 150 mayincrease the resolution of the final image to that of the first imageand display the final image with increased resolution on the imageoutput unit 160.

In addition to the first synthesis and the second synthesis describedabove, the image synthesis unit 150 may generate a final image bycorrecting brightness and color information of the first image withreference to brightness and color information of the second image(hereinafter referred to as “third synthesis”). Specifically, the imagesynthesis unit 150 may refer to brightness and color information of apixel in the second image and correct brightness and color informationof pixels of the first image used in the pixel of the second image,based on the brightness and color information referred to. For example,if the first and second images are as illustrated in FIGS. 3 and 4,respectively, the image synthesis unit 150 may refer to brightness andcolor information of the pixel R1 of the second image and correctbrightness and color information of the pixels of the first image thatform the pixel R1 of the second image, that is, the pixels r1, r2, r3and r4, based on the brightness and color information referred to.

The image output unit 160 may display a final image generated by theimage synthesis unit 150. The display unit 160 may be any type ofelectronic display, such as a liquid crystal display (LCD), a plasmadisplay panel (PDP), a light emitting diode (LED), an organic LED(OLED), or a flexible display.

Although not shown, the image generation system 100 may include astorage unit. The storage unit may store the first image generated bythe first image generation unit 141, the second image generated by thesecond image generation unit 142, and the final image generated by theimage synthesis unit 150, for example. The storage unit may furtherstore resolution information of the first image, resolution informationof the second image, and predetermined brightness information of thesecond image. In the above example, the first image, the second image,the final image, and the resolution information may all be stored in onestorage unit. However, the present invention is not limited thereto. Thefirst image, the second image, the final image and the resolutioninformation may be stored in respective storage units.

FIG. 6 illustrates an image generation method, such as used by the imagegeneration system 100 of FIG. 2.

The first exposure time for acquiring the first image may be calculatedin operation 610, e.g., by the calculation unit. The first exposure timemay be calculated, for example, using Equation (2).

Operation 610 may include an operation of calculating the secondexposure time T₂ for acquiring the second image with the predeterminedbrightness Y₂ based on the predetermined brightness Y₂ of the secondimage and an operation of calculating the first exposure time bydividing the calculated second exposure time T₂ by the number N ofpixels of the first image included in each pixel of the second image.

The shutter may be adjusted, e.g., by the exposure adjustment unit 120,during the first exposure time, e.g., calculated by the calculation unit110, or the time spent on receiving light by the optical sensor. Animage of a subject may be acquired, e.g., by the image acquisition unit130, based on the amount of light received during the first exposuretime, in operation 620.

Once the image is acquired, e.g., by the image acquisition unit 130, thefirst image may be generated from the acquired image in operation 630,e.g. by the first image generation unit 141. In the present embodiment,the first image may be understood as being identical to the acquiredimage.

If the first image is acquired, the second image, which has lowerresolution than the first image, may be generated from the first imagein operation 640, e.g., by the second image generation unit 142. Here,the second image may be generated according to a predeterminedresolution of the second image, e.g., by the second image generationunit 142. Specifically, if the resolution of the second image is 1/N ofthat of the first image, the second image may be generated using Npixels, which use an identical color filter, in the first image as onepixel, e.g., by the second image generation unit 142.

A final image may be generated by synthesizing the first and secondimages in operation 650, e.g., by the image synthesis unit 150. Whengenerating the final image by synthesizing the first and second images,the resolution of one of the first image and the second image may beadjusted to match the resolution of the other first and second imagesand the first and second images may be synthesized with matchingresolution, e.g., by the image synthesis unit 150. The first and secondimages may be synthesized using a first synthesis method or a secondsynthesis method. A method of generating a final image using the firstsynthesis method and a method of generating a final image using thesecond synthesis method will be described in greater detail later withreference to FIGS. 7 through 10.

In addition to generating the final image by synthesizing the first andsecond images with matching resolution, the final image may be generatedusing a third synthesis method, e.g., by the image synthesis unit 150. Amethod of generating a final image using the third synthesis method willbe described in greater detail later with reference to FIGS. 11 and 12.

The image output unit 160 may display the final image generated by theimage synthesis unit 150.

Hereinafter, a method of generating a final image using the firstsynthesis method will be described with reference to FIGS. 7 and 8.

FIG. 7 illustrates operation 650 of FIG. 6 in greater detail, assumingthat a final image is generated using the first synthesis method.

Referring to FIG. 7, resolution of the second image may be increased tothat of the first image while maintaining the resolution of the firstimage unchanged, so that the resolution of the second image can matchthat of the first image in operation 651, e.g., by the image synthesisunit 150. Here, the resolution of the second image may be increasedthrough, for example, linear interpolation. After increasing theresolution of the second image, the first image and the second image maybe synthesized with increased resolution and a final image may begenerated in operation 652, e.g., by the image synthesis unit 150.

FIG. 8 illustrates a process of generating a final image using the firstsynthesis method of FIG. 7.

Referring to FIG. 8, a first image 81 may be generated, e.g., by thefirst image generation unit 141, from an image, e.g., an image acquiredby the image acquisition unit 130, and a second image 82 may begenerated from the first image 81, e.g., by the second image generationunit 142, with resolution and brightness different from that of thefirst image 81. Since the first image 81 has higher contrast than thesecond image 82, its edges may be clearer. In addition, the color of thesecond image 82 may be clearer than that of the first image 81.

Next, the resolution of the second image 82 may be increased to that ofthe first image 81, e.g., by the image synthesis unit 150. Then, thefirst image 81 and a second image 83 may be synthesized with increasedresolution and a final image 84 may be generated, e.g., by the imagesynthesis unit 150. Here, since the contrast of the final image 84 isbetween the contrast of the first image 81 and the contrast of thesecond image 82, the final image 84 may have a clear color, as well as aclear edge. The final image 84 may be displayed, e.g., on the imageoutput unit 160.

Hereinafter, a method of generating a final image using the secondsynthesis method will be described with reference to FIGS. 9 and 10.

FIG. 9 illustrates operation 650 of FIG. 6 in greater detail, assumingthat a final image is generated using the second synthesis method.

Referring to FIG. 9, the resolution of the first image may be reduced tothat of the second image while maintaining the resolution of the secondimage unchanged so that the resolution of the first image can match thatof the second image in operation 656, e.g., by the image synthesis unit150. After reducing the resolution of the first image the second imageand the first image may be synthesized with reduced resolution and afinal image may be generated in operation 657, e.g., by the imagesynthesis unit 150. Then, the resolution of the final image may beincreased to that of the first image in operation 658, e.g., by theimage synthesis unit 150.

FIG. 10 illustrates a process of generating a final image using thesecond synthesis method of FIG. 9.

Referring to FIG. 10, a first image 91 may be generated, e.g., by thefirst image generation unit 141, from an image, e.g., an image acquiredby the image acquisition unit 130, and a second image 92 by generatedfrom the first image 91 with resolution and brightness different fromthe first image 91, e.g. by the second image generation unit 142. Here,the second image 92 may be generated from the first image 91, e.g., bythe second image generation unit 142, according to predeterminedresolution of the second image 92. Since the first image 91 has highercontrast than the second image 92, its edges may be clearer. Inaddition, the color of the second image 92 may be clearer than that ofthe first image 91.

Next, the resolution of the first image 91 may be reduced to that of thesecond image 92, e.g., by the image synthesis unit 150. Then, the secondimage 92 and a first image 93 may be synthesized with reduced resolutionand a final image 94 may be generated, e.g., by the image synthesis unit150. Since the resolution of the final image 94 may be reduced to thatof the second image 92, the resolution of the final image 94 may beincreased to that of the first image 91, e.g., by the image synthesisunit 150. A final image 95 with increased resolution may be displayed,e.g., on the image output unit 160.

Hereinafter, a method of generating a final image using the thirdsynthesis method will be described with reference to FIGS. 11 and 12.

FIG. 11 illustrates operation 650 of FIG. 6 in greater detail, assumingthat a final image is generated using the third synthesis method.

Referring to FIG. 11, brightness and color information of the secondimage may be referred to in operation 653, e.g., by the image synthesisunit 150. Then, the brightness and color of the first image may becorrected based on the referred brightness and color information of thesecond image and a final image may be generated in operation 654, e.g.,by the image synthesis unit 150. Specifically, brightness and colorinformation of a pixel in the second image may be referred to and thebrightness and color of pixels of the first image used in the pixel ofthe second image may be corrected, based on the brightness and colorinformation referred to.

FIG. 12 illustrates a process of generating a final image using thethird synthesis method of FIG. 11.

Referring to FIG. 12, a first image 71 may be generated, e.g., by thefirst image generation unit 141, from an image acquired, e.g., by theimage acquisition unit 130, and a second image 72 may be generated, fromthe first image 71, with resolution and brightness different from thoseof the first image 71, e.g., by the second image generation unit 142.Here, the second image 72 may be generated from the first image 71,e.g., by the second image generation unit 142, according to apredetermined resolution of the second image 72. Since the first image71 may have higher contrast than the second image 72, its edges may beclearer. In addition, the color of the second image 72 may be clearerthan that of the first image 71.

Next, brightness and color information of a pixel in the second image 72may be referred to, e.g., by the image synthesis unit. Then, thebrightness and color information of pixels of the first image, whichform the pixel of the second image, may be corrected according to thebrightness and color information referred to and a final image may begenerated, e.g. by the image synthesis unit 150.

In the above embodiment, an image generation system, method and medium,which calculate the first exposure time of the first image to acquirethe second image with predetermined brightness when the resolution ofthe second image is predetermined, have been described. Hereinafter, animage generation system, method and medium, which calculate resolutionof a second image generated from a first image when the exposure time ofthe first image is predetermined, according to another embodiment of thepresent invention will be described with reference to FIG. 13.

Since the operations of an image synthesis unit 250 and an image outputunit 260 illustrated in FIG. 13 are identical to those of the imagesynthesis unit 150 and the image output unit 160 illustrated in FIG. 2,a detailed description thereof will be omitted. Thus, the description ofthe present embodiment may be focused on the operations of an imageacquisition unit 230, a calculation unit 210, and an image generationunit 240.

The calculation unit 210 may calculate resolution of a second imagebased on a predetermined first exposure time, and a second exposuretime, for acquiring the second image with predetermined brightness.Specifically, the calculation unit 210 may divide the second exposuretime T₂ for acquiring the second image with the predetermined brightnessY₂ by the predetermined first exposure time T₁ as in Equation (3).

$\begin{matrix}{{N = \frac{T_{2}}{T_{1}}},} & {{Equation}\mspace{14mu} 3}\end{matrix}$

where N indicates the number of pixels of the first image included ineach pixel of the second image. For example, if N=4, each pixel of thesecond image may consist of four pixels, which use an identical colorfilter, in the first image. In other words, the resolution of the secondimage is a quarter of the resolution of the first image. Therelationship between the resolution of the second image and that of thefirst image may be given, for example, by Equation (1) described above.

The image acquisition unit 230 may acquire an image of a subjectaccording to the predetermined first exposure time.

Specifically, a first image generation unit 241 may generate the firstimage from an image acquired by the image acquisition unit. In thiscase, the acquired image and the first image may be understood asidentical images.

A second image generation unit 242 may receive the first image from thefirst image generation unit 241 and generate the second image from thefirst image based on resolution information of the second imagecalculated by the calculation unit 210. For example, if a value of N,which is calculated, for example, using Equation (3), is 4, the secondimage generation unit 242 may generate the second image by correspondingfour pixels of the first image, which use an identical color filter, toa pixel of the second image.

The image synthesis unit 250 may synthesize the first image generated bythe first image generation unit 241 and the second image generated bythe second image generation unit 242 and may generate a final image. Inthis case, the image synthesis unit 250 may synthesize the first andsecond images through the first synthesis or the second synthesis.Alternatively, the image synthesis unit 250 may correct brightness andcolor information of a plurality of pixels of the first image based onbrightness and color information of a referred pixel in the second imageand generates the final image.

FIG. 14 illustrates an image generation method used, for example, by theimage generation system 200 of FIG. 13.

Referring to FIG. 14, in operation 710, resolution of a second image maybe calculated, e.g., by the calculation unit 210, based on apredetermined first exposure time T₁ and a second exposure time T₂ foracquiring the second image with predetermined brightness Y₂, and thecalculated resolution of the second image may be provided, for example,to the second image generation unit 242.

A shutter may be adjusted, e.g., by the exposure adjustment unit 220, sothat light can be received during the first exposure time, e.g., by theimage acquisition unit 230. In the case of a camera sensor without amechanical shutter, an optical sensor may be allowed to receive lightduring the first exposure time, e.g., by the exposure adjustment unit220.

In operation 720, an image of a subject may be acquired based on anamount of light received during the first exposure time, e.g., by imageacquisition unit 230.

In operation 730, a first image may be generated, e.g., by the firstimage generation unit 241, from the acquired image, e.g., an imageacquired by the image acquisition unit 230. In the present embodiment,the first image may be understood as the acquired image. The first imagemay have resolution corresponding to the number of pixels included in aCIS of the image generation system 200 as illustrated in FIG. 3.

In operation 740, the second image may be generated e.g., by the secondimage generation unit 242, with lower resolution than the first imagefrom the first image generated e.g., by the first image generation unit241. Specifically, the second image may be generated, e.g., by secondimage generation unit 242, with reference to resolution informationprovided by the calculation unit 210. For example, if the resolution ofthe second image is a quarter of the resolution of the first image, thesecond image may be generated, e.g., by the second image generation unit242, such that each pixel of the second image may include four pixelsusing an identical color filter in the first image.

In operation 750, the resolution of the first and second images may bematched and the first and second images may be synthesized with matchingresolution, e.g., by the image synthesis unit 250. Here, the first andsecond images may be synthesized using the first synthesis method or thesecond synthesis method as in the previous embodiment. If the first andsecond images are synthesized using the second synthesis method, a finalimage may additionally be processed in order to increase the resolutionof the final image, e.g., by the image synthesis unit 250.Alternatively, brightness and color information of the first image maybe corrected based on brightness and color information of the secondimage referred to and a final image may be generated, e.g., by the imagesynthesis unit 250.

In operation 760, the final image generated by the image synthesis unit250 may be displayed, e.g. on the image output unit 260.

An image generation system, method and medium, according to one or moreembodiments of the present invention has been described with referenceto block diagrams or flowchart illustrations. It may be understood thateach block of the flowchart illustrations, and combinations of blocks inthe flowchart illustrations, can be implemented by computer programinstructions. These computer program instructions can be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing system, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, implement the functions specified in theflowchart block or blocks.

These computer program instructions may also be stored in a computerusable or computer-readable memory that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer usable orcomputer-readable memory produce an article of manufacture includinginstructions that implement the function specified in the flowchartblock or blocks.

The computer program instructions may also be loaded onto a computer orother programmable data processing apparatus to cause a series ofoperations to be performed on the computer or other programmableapparatus to produce a computer implemented process such that theinstructions that execute on the computer or other programmableapparatus implement the functions specified in the flowchart block orblocks.

And each block of the flowchart illustrations may represent a module,segment, or portion of code, which comprises one or more executableinstructions for implementing the specified logical function(s). Itshould also be noted that in some alternative implementations, thefunctions noted in the blocks may occur out of the order. For example,two blocks shown in succession may in fact be executed substantiallyconcurrently or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved.

In addition to the above described embodiments, embodiments of thepresent invention may also be implemented through computer readablecode/instructions in/on a medium, e.g., a computer readable medium, tocontrol at least one processing element to implement any above describedembodiment. The medium can correspond to any medium/media permitting thestoring and/or transmission of the computer readable code.

The computer readable code may be recorded/transferred on a medium in avariety of ways, with examples of the medium including recording media,such as magnetic storage media (e.g., ROM, floppy disks, hard disks,etc.) and optical recording media (e.g., CD-ROMs, or DVDs), andtransmission media such as carrier waves, as well as through theInternet, for example. Thus, the medium may further be a signal, such asa resultant signal or bitstream, according to embodiments of the presentinvention. The media may also be a distributed network, so that thecomputer readable code is stored/transferred and executed in adistributed fashion. Still further, as only an example, the processingelement could include a processor or a computer processor, andprocessing elements may be distributed and/or included in a singledevice.

As described above, an image generation system, method and medium,according to the present invention provide at least one of the followingadvantages.

First, since a plurality of images with different luminance can beacquired in a single photographing operation, a high dynamic range imagemay be generated.

Second, an exposure time may be reduced, thereby enhancing userconvenience.

Third, since the exposure time may be reduced, image blurring due to theshaking hands of a user during a long exposure time may be prevented.

Fourth, a high dynamic range image may be generated without modifyingthe structure of an image photographing system.

Although a few embodiments of the present invention have been shown anddescribed, it would be appreciated by those skilled in the art thatchanges may be made in these embodiments without departing from theprinciples and spirit of the invention, the scope of which is defined inthe claims and their equivalents.

What is claimed is:
 1. An image generation apparatus comprising: animage acquisition unit configured to acquire an image of a scene from asingle exposure; an image generation unit configured to generate, fromthe acquired image, a plurality of images of the scene, each with eitheror both of a different resolution and a different brightness; and animage synthesis unit configured to synthesize the plurality of generatedimages.
 2. The apparatus of claim 1, wherein the image generation unitcomprises: a first image generation unit configured to generate a firstimage; and a second image generation unit configured to generate asecond image, having lower resolution than the first image, from thefirst image.
 3. The apparatus of claim 2, further comprising: acalculation unit configured to calculate a first exposure time forobtaining the first image, based on a second exposure time for obtainingthe second image, with predetermined resolution and brightness.
 4. Theapparatus of claim 3, wherein the first image generation unit generatesthe first image from the image acquired by the image acquisition unitaccording to the calculated first exposure time.
 5. The apparatus ofclaim 2, wherein the first image generation unit generates the firstimage from the image acquired by the image acquisition unit according topredetermined first exposure time.
 6. The apparatus of claim 5, furthercomprising: a calculation unit configured to determine resolution of thesecond image based on the first exposure time and the second exposuretime for obtaining the second image with the predetermined brightness.7. The apparatus of claim 6, wherein the calculation unit determines theresolution of the second image to be a value obtained after a ratio ofthe first exposure time to the second exposure time is multiplied byresolution of the first image.
 8. An image generation method comprising:acquiring an image of a scene from a single exposure; generating, fromthe acquired image, a plurality of images of the scene, each with eitheror both of a different resolution and a different brightness; andsynthesizing the plurality of generated images.
 9. The method of claim8, wherein the generating of the images comprises: generating a firstimage; and generating a second image, having lower resolution than thefirst image, from the first image.
 10. The method of claim 9, furthercomprising calculating a first exposure time for obtaining the firstimage, based on a second exposure time for obtaining the second image,with predetermined resolution and brightness.
 11. The method of claim10, wherein the generating of the first image comprises generating thefirst image from the acquired image according to the calculated firstexposure time.
 12. The method of claim 9, wherein the generating of thefirst image comprises generating the first image from the acquired imageaccording to a predetermined first exposure time.
 13. The method ofclaim 12, further comprising determining resolution of the second imagebased on the first exposure time and the second exposure time forobtaining the second image with the predetermined brightness.
 14. Themethod of claim 13, wherein the determining of the resolution of thesecond image comprises determining the resolution of the second image tobe a value obtained after a ratio of the first exposure time to thesecond exposure time is multiplied by a resolution of the first image.